Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization

We propose the first, to our knowledge, loss function for approximate Nash equilibria of normal-form games that is amenable to unbiased Monte Carlo estimation. This construction allows us to deploy standard non-convex stochastic optimization techniques for approximating Nash equilibria, resulting in novel algorithms with provable guarantees. We complement our theoretical analysis with experiments demonstrating that stochastic gradient descent can outperform previous state-of-the-art approaches

A Generalized Training Approach for Multiagent Learning

This paper investigates a population-based training regime based on game-theoretic principles called Policy-Spaced Response Oracles (PSRO). PSRO is general in the sense that it (1) encompasses well-known algorithms such as fictitious play and double oracle as special cases, and (2) in principle applies to general-sum, many-player games. Despite this, prior studies of PSRO have been focused on two-player zero-sum games, a regime wherein Nash equilibria are tractably computable. In moving from two-player zero-sum games to more general settings, computation of Nash equilibria quickly becomes infeasible. Here, we extend the theoretical underpinnings of PSRO by considering an alternative solution concept, $\alpha$-Rank, which is unique (thus faces no equilibrium selection issues, unlike Nash) and applies readily to general-sum, many-player settings. We establish convergence guarantees in several games classes, and identify links between Nash equilibria and $\alpha$-Rank. We demonstrate the competitive performance of $\alpha$-Rank-based PSRO against an exact Nash solver-based PSRO in 2-player Kuhn and Leduc Poker. We then go beyond the reach of prior PSRO applications by considering 3- to 5-player poker games, yielding instances where $\alpha$-Rank achieves faster convergence than approximate Nash solvers, thus establishing it as a favorable general games solver. We also carry out an initial empirical validation in MuJoCo soccer, illustrating the feasibility of the proposed approach in another complex domain

Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments

The Game Theory & Multi-Agent team at DeepMind studies several aspects of multi-agent learning ranging from computing approximations to fundamental concepts in game theory to simulating social dilemmas in rich spatial environments and training 3-d humanoids in difficult team coordination tasks. A signature aim of our group is to use the resources and expertise made available to us at DeepMind in deep reinforcement learning to explore multi-agent systems in complex environments and use these benchmarks to advance our understanding. Here, we summarise the recent work of our team and present a taxonomy that we feel highlights many important open challenges in multi-agent research.Comment: Published in AI Communications 202

The Mitochondrial Acyl-carrier Protein Interaction Network Highlights Important Roles for LYRM Family Members in Complex I and Mitoribosome Assembly

NDUFAB1 is the mitochondrial acyl carrier protein (ACP) essential for cell viability. Through its pantetheine-4'-phosphate post-translational modification, NDUFAB1 interacts with members of the leucine-tyrosine-arginine motif (LYRM) protein family. Although several LYRM proteins have been described to participate in a variety of defined processes, the functions of others remain either partially or entirely unknown. We profiled the interaction network of NDUFAB1 to reveal associations with 9 known LYRM proteins as well as more than 20 other proteins involved in mitochondrial respiratory chain complex and mitochondrial ribosome assembly. Subsequent knockout and interaction network studies in human cells revealed the LYRM member AltMiD51 to be important for optimal assembly of the large mitoribosome subunit, consistent with recent structural studies. In addition, we used proteomics coupled with topographical heat-mapping to reveal that knockout of LYRM2 impairs assembly of the NADH-dehydrogenase module of complex I, leading to defects in cellular respiration. Together, this work adds to the catalogue of functions executed by LYRM family of proteins in building mitochondrial complexes and emphasizes the common and essential role of NDUFAB1 as a protagonist in mitochondrial metabolism

Accessory subunits are integral for assembly and function of human mitochondrial complex I

Complex I (NADH:ubiquinone oxidoreductase) is the first enzyme of the mitochondrial respiratory chain and is composed of 45 subunits in humans, making it one of the largest known multi-subunit membrane protein complexes. Complex I exists in supercomplex forms with respiratory chain complexes III and IV, which are together required for the generation of a transmembrane proton gradient used for the synthesis of ATP. Complex I is also a major source of damaging reactive oxygen species and its dysfunction is associated with mitochondrial disease, Parkinson's disease and ageing. Bacterial and human complex I share 14 core subunits that are essential for enzymatic function; however, the role and necessity of the remaining 31 human accessory subunits is unclear. The incorporation of accessory subunits into the complex increases the cellular energetic cost and has necessitated the involvement of numerous assembly factors for complex I biogenesis. Here we use gene editing to generate human knockout cell lines for each accessory subunit. We show that 25 subunits are strictly required for assembly of a functional complex and 1 subunit is essential for cell viability. Quantitative proteomic analysis of cell lines revealed that loss of each subunit affects the stability of other subunits residing in the same structural module. Analysis of proteomic changes after the loss of specific modules revealed that ATP5SL and DMAC1 are required for assembly of the distal portion of the complex I membrane arm. Our results demonstrate the broad importance of accessory subunits in the structure and function of human complex I. Coupling gene-editing technology with proteomics represents a powerful tool for dissecting large multi-subunit complexes and enables the study of complex dysfunction at a cellular level